1. Field of the Invention
This invention relates to a system for cataloguing, storing, retrieving, and distributing multimedia data.
2. Background
Increasingly, computer systems are being used to present multimedia material. Such material is usually in the form of text, graphics, video, animation, and sound. (Other examples include books, manuscripts, pictures, film, audio recordings.) Two or more of these data types are usually combined to form the multimedia data presented by the computer system. A computer system that is used to present multimedia material is called a multimedia system.
The multimedia material must be captured before it can be used by a multimedia system. For example, some or all of the multimedia data is in an analog format and must be converted to digital form before it can be used by a multimedia system. The multimedia data converted from analog must be managed. A management facility is needed to manage and maintain the data. An access capability is needed to search and retrieve stored multimedia data. Once it is retrieved, a distribution facility is needed to distribute the data to a multimedia system. Thus, a system is needed to capture, access, manage, and distribute multimedia material. Such a system is digital library system.
A number of patents that describe software and/or hardware systems are provided below. These systems do not provide a digital library system that can capture, access, manage, and distribute multimedia material.
A search system is described in U.S. Pat. No. 5,241,671, Reed et al., issued on Aug. 31, 1993 relates to a multimedia system that includes a database that is comprised of words, phrases, numbers, letters, maps, charts, pictures, moving images, animations, and audio information. A search capability is provided that provides a series of entry paths for locating information in the database. An entry path allows a user to enter a search request that consists of a set of valid terms or stop terms. A stop term is a term that exists on a stop term list and may be the words xe2x80x9cthexe2x80x9d or xe2x80x9caxe2x80x9d, for example. Valid terms are linked to related terms using a stem index. A stem index contains a root term and a set of stems for each term that is related to the root word. For example, the word leaf is linked to terms xe2x80x9cleavesxe2x80x9d and xe2x80x9cleafingxe2x80x9d.
A repetitive analysis event system that accesses data using a time-based number is described in U.S. Pat. No. 5,414,644, Seaman et al., issued on May 9, 1995. The system uses an information library that consists of visual data storage and a textual database for storing written descriptions of the visual data and a glossary of keywords that identify repetitive events or behavior. A behavioral label is used to define a behavioral activity. A series of images or video clips are associated with the behavioral label. A user can retrieve images by identifying a subject, a behavioral activity, or other type of descriptive text. A chronological timeline is used to control the order in which the images are displayed. That is, the images are displayed in sequential order using the time-based number.
A method for storing and retrieving information that uses a xe2x80x9cthree filexe2x80x9d concept is described in U.S. Pat. No. 3,670,310, Bharwani et al., issued on Jun. 13, 1972. This concept is so named because it uses three files: an index file, a search file, and a data file. The index file contains a keyword record for each keyword. A keyword is a word that exemplifies the meaning or value of information. The search file contains a search record for each data record in the data file. A search record contains an address for its associated data record in the data file, the keywords for the data record. Each keyword in the search record has an associated link address to another search record that contains the same keyword. Bharwani states that it improves on the xe2x80x9cthree filexe2x80x9d concept by: 1) using an adjustable keyword length in the index file to conserve space, 2) providing a means for marking items as deleted so that they can be bypassed, 3) automatically generating keywords based on field values contained in a data record, and 4) providing continuous searching even during a database update procedure.
A knowledge based information retrieval system is described in U.S. Pat. No. 5,404,506, Fujisawa et al., issued on Apr. 4, 1995. The system provides a visual interface for local searching and a natural language interpreter for global search. The natural language interpreter is used to infer the meaning of a noun phrase or a nominal phrase. The inferred meaning is used to retrieve information.
A system for distributed data from a centralized location to a remote location is described in U.S. Pat. No. 4,974,149, Valenti, Nov. 27, 1990. The centralized system generates a descriptor that describes the data and its source location. The descriptor may further define when the data is to be retrieved. The remote systems use the descriptor to retrieve the data. A preferred embodiment uses the descriptor to distribute software (i.e., executable and data files).
A system for database retrieval wherein entries in different databases are retrieved by a process of matching key words of the databases is described in U.S Pat. No. 5,210,868, Shimada et al., issued on May 11, 1993. Examples of two such databases are a mapping database and a customer attribute database. A dictionary is used to separate a keyword from a first database into common and proper noun subparts. Common and proper noun synonyms are inferred according to a set of rules. The synonyms are combined using a combination rule and then compared with keywords in a second database to generate a final matching result.
A system for assembling documents, or files is disclosed in U.S. Pat. No. 5,222,236, Potash et al., issued on Jun. 22, 1993. Documents are stored in a library that is composed of a series of volumes. A volume is a logical grouping of documents such as legal documents. A volume is decomposed into a set of chapters. For example, a chapter within a xe2x80x9clegal documentsxe2x80x9d volume might contain documents related to commercial law. The system further provides a series of user interface screens that prompt a user for input that is used to update a data field in a document (e.g., addressee).
A process for creating a search query from a natural language query is described in U.S. Pat. No. 5,265,065, Turtle, issued on Nov. 23, 1993. A database contains a plurality of stopwords and phrases. The database is used to eliminate stopwords from the natural language query. The remaining words are stemmed to their basic roots, or words. The stemmed words are compared against the database to identify phrases. The phrases and any remaining stemmed roots form the search query and are referred to as query nodes of a query network. The query nodes are also representation nodes of a document. A synonym database is used for finding synonyms.
A service for managing hypertext/hypermedia links is described in U.S. Pat. No. 5,297,249, Bernstein et al., issued on Mar. 22, 1994. A Link Manager Services (LMS) component provides a universal End User Interface (EUI) including menus and dialog boxes that can be called by an application program when a link, or marker, is selected by an application user. The LMS accesses a database that contains information about markers and their associated xe2x80x9cpresentersxe2x80x9d (e.g., application or program). In addition, the LMS includes a viewer capability for viewing and maintaining existing links and creating new links. The LMS is used at runtime to generate menus and dialog boxes and manage the hypertext/hypermedia links.
A system that interconnects audio-video equipment such as video tape record or video production switcher using one or more xe2x80x9cAV LANsxe2x80x9d in U.S. Pat. No. 5,307,456, Mackay, issued on Apr. 26, 1994. Generic device commands are communicated over an AV LAN to a device translator. The device translator translates the generic command into a device-specific command. The AV LANs are interconnected using bridges. The AV LANs can further be used to interconnect workstations, minicomputers, mainframes and personal computers. The workstations can be used to display information about the resources attached to an AV LAN.
A text searching system that includes: 1) a morphological analyzer that generates a set of words that are lexically related to an input word and that together with the input word constitute a set of search words and 2) a search engine for searching an index to detect the occurrence of any of the search words in the index is described in U.S. Pat. No. 5,369,577, Kadashevich et al., issued on Nov. 29, 1994. The morphological analyzer includes a recognition engine that identifies derivational information for a given input word. A thesaurus database is used to generate a list of synonyms for the lexically related words. The synonyms and variations of the synonyms (i.e., words formed by adding a suffix to a synonym) are included in the set of search words. The system can further be used to expand an input word into related words by removing suffixes. Further expansion is accomplished by using the thesaurus database to obtain synonyms for the related words.
A system for dynamically loading software libraries. A service requester of an application program sends a request to a loader module of an application program is described in US. Pat. No. 5,410,698, Danneels et al., issued on Apr. 25, 1995. The application""s loader module forwards the request to a second loader module outside the application program. The service requester can be a media services manager of a multicast application program that issues a request to load a media service provider library. A global dynamic loader receives the request as part of the multicast application program and forwards it to the global dynamic loader executable (i.e., the second loader module) that loads the media service provider library.
A method for manipulating video data that allow for the definition of frames in a string of isochronous data using frame numbers or names is described in U.S. Pat. No. 5,414,808, Williams et al., issued on May 9, 1995. Frames can be grouped into segments. Frames can be referenced and operations performed on frames using the frame definitions.
A method for storing data in data stores at various locations in a network is described in U.S. Pat. No. 5,442,771, Filepp et al., issued on Aug. 15, 1995. These locations can be the user system and network concentrator facilities hierarchically located between the user system and the network host. Data is cached in either a temporary cache (e.g., RAM) or a variable-content permanent stage (e.g., variable-content, fixed disk file). A least-recently-used approach is used to determine the data that is retained in a store. In addition, storage candidacy and version control parameters are used to insure currency if maintained for time-sensitive data (e.g., news or pricing data).
An image data filing system consisting of a library for storing a plurality of image storage media (e.g., optical disks), a disk array for storing image data retrieved from the image storage media, a console for entering user instructions, and an output device for displaying image data is described in U.S. Pat. No. 5,463,771, Sotoyanagi et al., issued on Oct. 31, 1995. A control device is used to control the retrieval and storage operations.
A system for storing and retrieving digital images is described in U.S. Pat. No. 5,493,677, Balogh et al., issued on Feb. 20, 1996. A caption or other metadata can be associated with a digital image. A natural language capability removes ambiguities from the metadata input by a user prior to its storage. The natural language capability determines matches between a user query and the stored metadata. The system allows a user to select an image, review licensing terms for the selected image, and order the image.
A technique for dynamically optimizing the processing of a database query execution plan is described in U.S. Pat. No. 5,495,608, Antoshenkov et al., issued on Feb. 27, 1996. A decision is made during processing to either retrieve records using indexed keys or by retrieving all records based on which approach is optimal.
The invention is a digital library system that includes: 1) a data capture mechanism that includes data transfer and cataloguing mechanisms, 2) an asset management system for access and storage management of data, and 3) a distribution system for distributing the data and system functionality.
A data capture system includes a transfer system and a cataloguing system. The transfer system converts multimedia material that exists in analog form to a digital format. In addition to the data output from the transfer system, the cataloguing system can also receive textual data as input. The cataloguing system catalogues the multimedia and textual data. The cataloguing system creates a catalogue that can be used to perform content-based searches. A content-based search retrieves data based on the ideas or concepts contained in the data. Data is stored by the transfer system independent of content. The cataloguing systems creates a catalogue that specifies the content of the data. The catalogue includes one or more catalogue elements which can be complex multimedia assets. A complex multimedia asset can consist of one or more attribute elements. An attribute element is an attribute that can have attributes (i.e., pieces of information). A catalogue element is associated with a portion of multimedia data (e.g., one or more frames of video data).
An asset management system is used to access the data using the catalogue created by the cataloguing system. The asset management system identifies data having the desired content. The asset management system includes search tools to query the catalogue to identify a set of catalogue elements having attributes that satisfy a search criteria. The data associated with the catalogue elements have the desired content.
The asset management system includes a browser, an indexing server, an archive server, a tertiary storage manager, and a method player. Generalized interfaces define a communication protocol that can be used by any vendor-supplied browser, indexing server, archive server, tertiary storage manager, and/or method player to communicate with another asset management system component.
A request for data is received by a browser. The browser in combination with the indexing server locates a set of catalogue elements that satisfy criteria specified in the request. The set of catalogue elements are sent to an archive server for retrieval. The archive server maintains an identification of the location of the multimedia data. Thus, when a set of catalogue elements is received from the browser, the archive server can identify the location of the portions of multimedia data having the desired content (i.e., the portions of multimedia data associated with the catalogue elements contained in the set).
The multimedia data associated with a catalogue element is retrieved by the archive server in conjunction with a tertiary storage manager. A tertiary storage manager manages a store that contains multimedia data. The system can consist of multiple instances of tertiary storage manager to manage each data store. The tertiary storage manager can manage various types of data store. The archive server identifies the tertiary storage manager that manages the store containing the requested multimedia data. The tertiary storage manager retrieves the data and transmits the data to a method player. The method player prepares the data (converts the data from MPEG to video format) and transmits the prepared data to the browser. The browser formats a display region and displays the data in the display region.
A caching mechanism is implemented by the asset management facility to store retrieved data locally. Preferably, cache is used to store multimedia data that is being or will be played on the browser. The asset management facility locates the data and, if necessary, stores the data in local cache. Preferably, cache management is supplied by an instance of tertiary storage manager. The tertiary storage manager that manages the data denormalizes the data allowing for faster access.
A tertiary storage manager that manages the cache uses a least recently used (LRU) scheme. Thus, multimedia data that has the oldest access time is purged to make room for newly accessed data. The archive server determines whether the local cache contains the multimedia data associated with the catalogue elements. If the multimedia data is not stored in local cache, the archive server can query other archive servers to determine whether the data is stored in a remote cache. If the archive server cannot find a cached copy of the data, it will attempt to retrieve the data from permanent storage (e.g., a tape system).
A distribution facility can be used to transmit the data thus giving a user access to all of the data contained in the digital library system despite the user""s location. Multimedia data is permanently stored at a centralized location. Multimedia data that is requested by a user is cached from the centralized location to the user site. A wide area network can be used to interconnect user sites with the main site. The WAN can be used to transmit data that resides at the main site or another user site to a requesting user site. In addition, data can be transferred between sites via the Internet.
Either a centralized or decentralized distribution architecture can be used with the invention to distribute functionality (e.g., asset management functionality). In a centralized architecture, the asset management facility is located at a centralized site. Users connect to the asset management facility via the Internet, for example. The catalogue and multimedia data are stored at the central site. The catalogue is accessed at the central, or main site to identify the requested data. The data is then transmitted to the user via the Internet.
In a decentralized architecture, an instance of the asset management system is located at the user""s site. A copy of the catalogue is resident at the user site. The multimedia data is permanently stored at a main site with copies of data that has accessed stored at the user""s site. The local catalogue is accessed to identify the data requested in a user request. If a copy of the requested data does not exist at the local site, a search is made for the data at another site. The search first examines the cache at the other sites. If the data cannot be found in cache at the local or another site. The local site accesses the main site to retrieve the requested data. A vehicle such as a WAN or the Internet can be used to transmit the data between sites.
The asset management system can include various access levels (i.e., simple to complex) each of which is associated with a level of interface to access the data based on the type of audience. The audience can be an individual or a group of people, for example. At the complex access level, the audience must be able to understand the catalogue structure (i.e., catalogue element and associated attributes and attribute elements).
At a programmer""s level, an interface consisting of low-level software routines are available to query the catalogue and its associated attributes and attribute elements. The low-level software routines can be used to create a higher level of abstraction of objects. These objects can become selectable elements in a graphical user interface (GUI) for a development tool that can be used to create a higher-level interface. The GUI can include a drag and drop capability to create a GUI and associate GUI objects to objects that supply functionality (e.g., objects contained in a higher-level interface.)
A development tool can be used to create a browser, for example. A browser can include the ability to view instances contained in a catalogue and its associated attributes and attribute elements. The user can create a query by dragging these instances into a search box and specifying conjunction information (e.g., xe2x80x9candxe2x80x9d or xe2x80x9cor).
At the simplest access level, predefined sets are created and stored for viewing by individuals using a single video viewing station or in a virtual reality environment or video display kiosks for viewing by groups of people.